User interfaces (UIs) for software programs are often designed to include UI elements that are easily selectable using mouse and/or keyboard input devices. However, the evolution of computer hardware has introduced several new input devices, including multi-touch display surfaces seen on smart phones and tablet personal computers (PCs). Though these multi-touch display surfaces are favorable in many aspects when interacting with multi-touch based UIs (which include large UI elements that are easy to select using one or more fingertips), the same does not apply when interacting with mouse-and-keyboard based UIs, which may be generated for display to the user on his or her computing device when the user is remotely accessing or running locally an application that is configured for mouse-and-keyboard based commands.
When the user is remotely accessing an application that is configured for mouse-and-keyboard based commands, the user controls a particular “host” machine remotely from his or her computing device. This type of interaction is commonly referred to as a “remoting” session. In typical remoting sessions, the user's computing device comprises input/output (IO) hardware that is similar to that of the host machine, such as a keyboard, a mouse, and a monitor. Throughout a remoting session, the video output data at the host machine is routed to the user's computing device, where such video data often includes a user interface (UI) of an operating system executing on the host machine. Conversely, user input data received at the user's computing device is routed to the host machine, where the host machine interprets the routed input data as if the input data was received locally at the host machine. For example, the user could select, using a mouse of his or her computing device, an executable (.exe) file of a software program located in a folder that is displayed within the UI of the host machine. In this example, the mouse command is routed from the remote machine to the host machine, where the mouse command causes the host machine to execute the selected software program. Then, the host machine routes new video data (which displays a UI of the executed software program) back to the remote machine.
However, if the user's computing device that is carrying out the remoting session does not employ a mouse and a keyboard but instead relies on a multi-touch display surface for receiving user inputs, user navigation of the UI becomes cumbersome and often results in inaccurate inputs. For example, a mouse click on a mouse-and-keyboard based UI specifies a particular (x, y) coordinate of the UI on which a cursor associated with the mouse currently lies. In contrast, tapping a finger to a multi-touch display surface implicates a plurality of (x, y) coordinates of the UI on which the tap is placed. Thus, when navigating a mouse-and-keyboard based UI using a finger, an approximation of the plurality of (x, y) coordinates into a single (x, y) coordinate is continually required. These approximations often lead to input inaccuracies, thereby reducing the overall productivity of the user when the user is remoting into or running locally an application that is configured for mouse-and-keyboard based commands.
Several modifications in UI design have been attempted to improve input accuracy in such situations. For example, the ability to zoom-in and zoom-out on a UI allows a user to interact more efficiently with any small interface elements included therein. However, such a UI technique requires that the user continually zooms-in and zooms-out on the UI, which is very time-consuming and represents a patchwork solution at best.